Parameterized Exercises in Java Programming: using Knowledge Structure for Performance Prediction

In this paper, we study the effect of using domain knowledge structure on predicting student performance with parameterized Java programming exercises. Domain knowledge structure defines connections between elementary knowledge items. While known to be beneficial in general, it has not been used to predict performance. We compare five different approaches for this purpose: Bayesian Knowledge Tracing (BKT), Performance Factor Analysis (PFA), and three dimensional Bayesian Probabilistic Tensor Factorization (3D-BPTF), that are not able to take into account knowledge structure; and four-dimensional Bayesian Probabilistic Tensor Factorization (4D-BPTF) and Feature-Aware Student Knowledge Tracing (FAST), that can take into account knowledge structure. We approach the problem using both topic-level and question-level Knowledge Components (KCs) and test the methods on a dataset of parameterized questions. Our work is the first in the field that models students’ behavior in a four dimensional tensor. Our experiments show that, when having only the knowledge-item-level information, all of the models work similarly in predicting student performance, but adding the topic-level information that integrates knowledge items changes the performance of these models in different directions.